chore(test-suite): sacrificial merge-queue validation for #2066#2087
Closed
chore(test-suite): sacrificial merge-queue validation for #2066#2087
Conversation
2 similar comments
🧪 CI InsightsHere's what we observed from your CI run for 26f5187. 🟢 All jobs passed!But CI Insights is watching 👀 |
Replace the 500-line bash fhevm-cli with a structured Bun/TypeScript implementation. The new CLI resolves version bundles from GitHub releases or GitOps branches, generates compose overrides and env files under .fhevm/, and boots the stack through a 13-step pipeline with resume support. Key modules: - versions.ts: resolve targets (latest-main, latest-release, gitops) - artifacts.ts: compose/env generation, local build isolation - runtime.ts: boot pipeline, test runner, pause/unpause - compat.ts: legacy CLI flag injection for older image versions - cli.test.ts: unit tests for version resolution, compat, layout Local builds use :fhevm-local image tags to avoid overwriting registry images. Services sharing an image tag are deduplicated before buildx.
…HEAD The previous approach copied .git/HEAD into the build context for version metadata. This breaks when building from a worktree or when .git is not a directory. Use a Docker build arg instead.
Add ARCHITECTURE.md with a mermaid boot-flow diagram and notes on version override, resume, and discovery. Add test-suite/fhevm/README.md with usage examples and env var override docs. Update the top-level test-suite/README.md to reference the new CLI commands.
Add Bun and Foundry setup steps. Replace ./fhevm-cli deploy with ./fhevm-cli up --target latest-release. Move version env vars to job-level env so the CLI picks them up as overrides. Show failure logs only on failure (not always). Add timeouts to log collection.
Add 57 new tests (65 total) covering utils, artifacts, CLI argument validation, command error paths, compat policy edge cases, and version resolution. Extract shared test helpers to reduce duplication and ensure safe cleanup of global state (process.exitCode, console captures).
These scripts are fully replaced by the Bun-based fhevm-cli and have zero references from CI workflows, docs, or other code.
* feat(test-suite): add per-service override support to fhevm-cli * feat(test-suite): warn user when selecting services that share a database
The merge with main incorrectly took the old bash script instead of the thin Bun wrapper entrypoint.
…rsions conservatively Three fixes for network target compatibility (devnet/testnet/mainnet): - Auto-detect MinIO key prefix during KMS signer discovery by probing both PUB/PUB (KMS <= v0.12) and PUB (KMS >= v0.13), then propagate the detected prefix through state for all MinIO URL construction. - Fix Docker volume mount conflicts where named volumes shadowed bind mounts to the same container path, preventing GatewayAddresses.sol from being accessible on the host. - Treat unparseable SHA versions conservatively in compat layer — apply legacy flags rather than skip them, since we cannot determine the actual version from a SHA tag.
Test-suite SDK < v0.11.0 (e.g. mainnet's 29ed8a7 with SDK 0.3.0-6) appends /v1/ internally to RELAYER_URL. SDK >= v0.11.0 (0.4.0) expects the URL to already include /v2. Set the base URL bare in the template and append /v2 only for modern test-suite versions.
Overriding a component on devnet/testnet/mainnet can cause DB schema or contract interface mismatches if local code diverges from the pinned versions. Add a non-blocking warning so devs are aware.
Destructive compat rewrites (relayer readiness config, RELAYER_URL path) must treat unparseable SHAs as modern, not old. Applying destructive rewrites to modern services removes config fields they require, causing crashes in workspace CI where images use SHA tags. Introduces versionLtStrict (SHA→modern) alongside existing versionLt (SHA→old) and uses it for the two destructive callers.
## Purpose Reduce reviewer cognitive load by eliminating duplication and dead code across three files (-34 lines, 3 concepts removed, 0 added). ## Structural changes - Merge versionLt/versionLtStrict into one function with unknownIsOld param - Extract defaultEndpoints(deps) to replace 4 copies of the endpoints literal - Delete unused companionPreset export from presets.ts ## Preserved contract - All 107 unit tests pass unchanged (no public signatures changed) - Additive compat rules still treat SHAs as old (unknownIsOld=true default) - Destructive compat rules still treat SHAs as modern (unknownIsOld=false) ## Suggested review order 1. compat.ts — versionLt parameterization 2. runtime.ts — defaultEndpoints helper + 4 call sites 3. presets.ts — dead export removal ## Risk areas - None: pure simplification, no behavioral change
requiresMultichainAclAddress used conservative SHA handling (SHA=old), which demanded MULTICHAIN_ACL_ADDRESS from gateway discovery even when the gateway contracts are modern and don't deploy it. Switch to unknownIsOld=false since requiring a non-existent address crashes the validate step (destructive). Also fix typo: unparseable → unparsable.
Workspace builds use SHA version tags (e.g. fd3ef74) which are unparsable by the semver comparator. Previously, unparsable versions were treated as "old" for additive compat rules, injecting legacy CLI flags like --coprocessor-api-key and --delegation-fallback-polling. Modern binaries built from HEAD reject these removed flags, crashing the service. Remove the unknownIsOld parameter from versionLt — all unparsable versions are now treated as modern. This is correct because SHA versions only appear in workspace builds which always use latest HEAD binaries.
- Delete copyFile (zero callers) and toError (unnecessary instanceof check in catch blocks where caught value is always Error) - Delete baseCompose (zero callers) - Simplify explainGitHubCliError: fewer substring checks, same 3 error categories (ENOENT, auth, rate limit) with same messages - Inline repoPackageName (1-caller wrapper around decodeURIComponent) - Simplify dockerInspect: early return on non-zero exit instead of parsing empty stdout with silent fallback - Simplify ensureCommand: use allowFailure instead of try/catch
Registry SHAs (--target sha) need compat flags because the image could be genuinely old, but workspace SHAs (local override) are always HEAD and reject legacy flags. Pass !hasOverride as unknownIsOld per group.
Reverts override-aware approach — the orchestrate CI uses registry mode with SHA-tagged images from main tip (no workspace overrides), so the override signal cannot distinguish fresh SHAs from old ones. Presets already map SHA targets to MODERN_COMPANIONS; compat now agrees.
Changed Lines CoverageCoverage of added/modified lines: N/A Per-file breakdownDiff CoverageDiff: origin/main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. |
Detect relayer v1/v2 API mismatch at validate time (fail fast instead of runtime 500s) and catch containers that crash shortly after boot by inspecting exit codes after a delay.
…le source of truth Move scattered version compatibility knowledge (relayer pins, ACL cutover SHA, legacy shims, incompatibility rules) into one declarative COMPAT_MATRIX in compat.ts. Add `compat-defaults` CLI subcommand so CI workflows read pins and anchors from the CLI instead of hardcoding them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a sacrificial draft PR to exercise the merge-queue-style E2E orchestration for #2066.
Context:
#2066:9ec15bd7test-suite-orchestrate-e2e-testsOnce the workflow signal is clear, this draft PR should be closed.